Peripheral Component Interconnect (PCI) Express implements a serial, switched-packet, point-to-point communication standard at a system level. The basic PCI Express architecture includes a root complex, a PCI Express switch chip, and various external devices. Communication packets in PCI Express include but are not limited to transaction layer packets (TLP).
PCI Express switches decode inbound transaction layer packet (TLP) headers and use corresponding fields in configuration space Base Address Registers, Base/Limit registers, and Bus Number registers to apply address, identification (ID), and implicit routing coding to the packet. There are two levels of decision: the device first determines if the packet targets an internal location, meaning it is intended for the device itself; if not, and the device is a switch, it evaluates the packet to see if it should be forwarded out of an egress port. If the packet has been received in error or is not properly formed, it is handled as a receive error. There are a number of cases when this may happen, and a number of ways it may be handled.
In a PCI Express switch, three address routing methods use Base Address Registers (BARs), and base address matching, to determine how to route incoming packets. The configuration of each port includes three BARs for the three addressing modes. A BAR includes a base address and a limit address as well as bus and port identifiers. If the target address in the header of a transaction layer packet is within the range of a downstream port's base and limit registers, the transaction layer packet is routed toward that port. If the target address is outside of the range of the upstream port's base and limit registers, the packet is routed toward the upstream port. If neither of these conditions are met, the packet has no route and is considered an unsupported request. If multiple ports detect a range match, such as can happen with an erroneous configuration, the behavior of the switch is not defined. This is referred to as a catastrophic routing condition or as a catastrophic route.
A catastrophic route means there is no correct, unambiguous, answer to which port a TLP should be routed. A catastrophic route can occur due to a design error in the switch or by a configuration error in the base address registers and IDs. If a switch behaves unpredictably due to a catastrophic route, it is possible for damage from a programming error to propagate in an unpredictable way. For example, mis-configuring one downstream port during a complex dynamic reconfiguration of that port, due to a hot-plug event for example, has the potential to disrupt operations of the entire network in which the switch is implemented. One result of catastrophic routing can be a hang-up, or stoppage, of the switch. By stopping switch operation and not accepting or reading further incoming packets, the switch is unable to accept and read a reconfiguration packet from the root complex that would cure the underlying problem. The only cure, in some instances, is to power down and restart the switch, which sometimes means restarting the entire system, so that it can receive a fresh reconfiguration.
Thus, there is a need for a method and apparatus that will detect catastrophic routes in a serial packet switch and deal with them so as to prevent disruption of switch operations. The method and apparatus of the present invention corrects such occurrences without disrupting network operations.